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Abstract 

In this paper, we consider the bacterial point-to-point communication problem with one transmitter 
and one receiver by considering the ligand receptor binding process. The most commonly investigated 
signalling model, referred to as the Level Scenario (LS), uses one type of a molecule with different 
concentration levels for signaling. An alternative approach is to employ multiple types of molecules 
with a single concentration level, referred to as the Type Scenario (TS). We investigate the trade-offs 
between the two scenarios for the ligand receptor from the capacity point of view. For this purpose, we 
evaluate the capacity using numerical algorithms. Moreover, we derive an upper bound on the capacity 
of the ligand receptor using a Binomial Channel (BIC) model using symmetrized Kullback-Leibler (KL) 
divergence. A lower bound is also derived when the environment noise is negligible. Finally, we analyse 
the effect of blocking of a receptor by a molecule of a different type, by proposing a new Markov model 
in the multiple-type signalling. 


I. Introduction 

Molecular communication (MC) has stimulated a great deal of interest because of its potential broad 
applications. There are different mechanisms for molecular communications, among which diffusion is 
the most favorable, as it does not require any prior infrastructure. In diffusion-based systems, information 
is encoded into the concentration, type or releasing time of the molecules. For instance, in |jT|, an on-off 
keying modulation is proposed where molecules are released only when the information bit is one. It is 
shown that if there is no interference from the previous transmission slots, the channel can be modeled by 
a Z-channel. In Q, Q, new modulation techniques based on multiple types of molecules are presented. 

Two models for diffusion-based channels have been proposed, namely small and large scales. Diffusion 
process is viewed as a probabilistic Brownian motion in the small scale model, whereas it is described 
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by deterministic differential equations in the large scale model. In this paper, we concentrate on the large 
scale model which reflects the average effects of diffusion. However, to derive the large-scale diffusion 
capacity of MC, one has to deal with the reception process at the receiver side. Two reception models 
are considered for a passive receiver. The first model is a perfect absorber where the receiver absorbs 
the hitting molecule. The second model, which is more realistic, is the ligand-receptor binding receiver, 
where the hitting molecule is absorbed by the receptor with some binding probability, 0, 0. Authors 
in 0 interpret the randonmess of ligand binding as noise and employ Markov chains to derive a closed 
form for it. Ligand receptors are modeled by a Markov chain in 0, by a discrete-time Markov model 
in 0, and by a Binomial channel for a hacterial colony in 0. The Binomial channel is defined hy 
p{y\x) = (px^(l — where the input x G [0,1], the output y G {0,1,..., n} and n, the number of 

trials, is a given natural number. Average and peak constraints on the input x may exist. The capacity of 
this channel without average and peak constraint, for large values of n behaves as follows 0: 


1 71 

T log -^ log ^ 

2 27re 


( 1 ) 


However there is no explicit upper or lower bound on the Binomial channel capacity when n is finite. 


An algorithm for computing the capacity of Binomial Channel was presented in |T0| using convex 
optimization methods. 

Our main contrihutions are as follows: 

• We investigate the tradeoffs between two bacterial communication scenarios for ligand receptors: (a) 
multi-type molecular communication with a single concentration level, and (h) single-type molecular 
communication via multiple concentration levels. At the first glance, scenario (a) introduces a new 
degrees of freedom and reduces the intersymbol interference (ISI). However since the number of 
molecules per type (the power per type) reduces by the increase of the types, we should examine 
the benefit of using different types of molecules. To make the comparison between scenario (a) and 
(b), we adopt the model of 0 in this work. 

• We derive an upper bound to the capacity of Binomial channel model with given average and 
peak constraints on the channel input, using KL divergence bound of | [TT| (Theorem [T]). Based on 
numerical evidence, we believe that this upper bound works well in the low SNR regime (which 
can occur in MC systems). 

• A lower bound for the Binomial channel with a peak constraint and no environment noise is presented 
in Lemma [T] Based on numerical results, we believe that this lower bound is tight for low peak 
values. 
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• A Markov model for the interactions between different types of molecules near the receptor is 
presented and the capacity for this model is computed numerically. 

All logarithms are in base e in this paper. This paper is organized as follows: in Section we 
present the system model for Level and Type signalling scenarios, whose the capacities are discussed in 


Subsection II-A In Section III a new upper bound on the capacity of the Binomial channel is presented 
by considering peak and average constraints. Section [TV] includes a lower bound on the capacity of the 
Binomial channel by extending the Z-channel. In Section |Vj the interaction of molecules near the receptor 
is modeled. Section includes the numerical results. 


II. System Model 

In this section, we describe two bacterial point-to-point communication scenarios with the ligand 
receptors, as shown in Fig. [T] The most commonly investigated model, referred to as the Level scenario 
(LS), uses one type of molecule with multiple concentration levels for transmission. An alternative 
approach is to provide different types of molecules with one concentration level, albeit at a higher 
cost, referred to as the Type scenario (TS). 

Level Scenario (LS): here the transmitter encodes information at multiple concentration levels to 
create the codewords. At the transmitter and the receiver, there is only one colony with n bacteria where 
each bacteria has N receptors; i.e., nN receptors in total. All these n bacteria produce just one type of 


molecule. This scenario is shown in Fig. 2b 


Type Scenario (TS): This scenario uses multiple types of molecules at the transmitter and the receiver. 
We assume the same total number of n bacteria (as in LS) are available which are equally divided into m 
colonies at both the transmitter and receiver as shown in FigJ^ As such, each colony has n/m bacteria. 
Moreover, different colonies at the transmitter produce different types of AHL molecules. Similar to 
the LS, each bacteria has N receptors. Furthermore, the colonies are synchronized at the transmitter. 
Therefore, there are nN/m receptors in total per each colony, i.e., each type of molecule. Each colony 
can detect its own molecule type and as a result, produces different color Fluorescent Proteins (e.g. 
GFP,YFP,...) which are used by the receiver to decode the received signal. In addition, we assume all 
receptors of a colony are independent and sense a common molecule concentration. 

Throughout Sections [T^ [11^ and IV we assume that the binding processes of different molecule types are 


independent. We investigate a more general model in Section IV by taking into account the interaction 
of different types of molecules at the TS. In both scenarios, we assume that there is no intersymbol 
interference (ISI). In other words, we assume those molecules, who do not bind to the receptors in the 
current time slot, will be degraded to the next time slot and hence will not interfere with molecules 
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Fig. 1: The system model 


from the next slot. This assumption, together with the large-scale diffusion channel property, result in a 
linear channel. For simplicity, we further assume that no attenuation occurs in the channel. Therefore, the 
received average concentration Ar is equal to the transmitted average concentration Ag. At the receiver 
with ligand receptors, the prohahility of binding at the steady state, is given hy Q: 

A. 


Pb = 


A.+ 5 


( 2 ) 


where 7 is the input gain and k is the dissociation rate of trapped molecules in the cell receptors. If we 
consider an environment noise with average concentration Am, due to the molecules of the same type 
from other sources, the prohahility of binding becomes: 

As + Ane 


Pb = 


As + Ane + ^ 


In LS, we only have one type of molecule and its binding probability is equal to 

Xls + A^f 


Pbhs 


0 < Xls < As 


(3) 


(4) 


Xls + ALS + ^ 

where Xls is the received average concentration at the receiver and A^f is the average concentration 
of environment noise. On the other hand, in TS, we have different types of molecules; the probability of 
binding for the ith type of molecule is given by 


xr + A 


i,TS 

ne 




0 < Xls ^ 


As 


m 


(5) 


where Xf^ is the received average concentration of the ith type of molecule at the receiver and Am^ 
is the average environment noise for the Ah type of molecule. Without loss of generality, we assume all 
environment noises have equal averages, i.e.. Am = A^g . Here, we consider the same 7 and k for all 
types of molecules and receptors. 


DRAFT 









5 




(a) Type scenario (TS) 


Channel 






Receiver 


(b) Level scenario (LS) 


Fig. 2: Two scenarios: TS and LS 


A. Capacity analysis 

In both scenarios, the output is discrete. Further, we assume an environment noise and consider average 
and peak signal concentration level constraints. The channel is Binomial as follows: 

p{Y = y\X = x)= (6) 

+ - /p,(x + A„e))^’""^, 

fpt ■ [0>oo] [0,1], y G {0, l,...,A^n} 

Note that for LS we have Am = A^f but for TS we have Am = A'^^ as the environment noises. The 
function fp^{X + Am) is the binding probability function where X is the signal concentration level and 
Am is the average environment noise. We also assume that the function fp^ is an increasing function and 
concave. We consider the function fp^{X + Am) = ligand receptors. As we note, when 

the concentration level increases, the binding probability also increases. 

In LS, we have a single colony with input X and output Y, with the following peak and average 
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concentration constraints: 

0<X < As 
E[X] < a 


(7) 

( 8 ) 


Then we find the capacity for LS as 

Cls = 


max 

0<X<a}, ^'E[X]<a 


liX-Y), Y G{0,l,...,Nn}. 


(9) 


In TS, we use Xi to denote the input of the fth colony to the channel and Yi to denote the output of 
the zth colony at the receiver. The constraints for TS are as follows 


, »/€/, 


m, 


0<Xi< — i = l 
- m 

CM. 

E[Xi] < , i = l, 

m 


Hence, the capacity can he written as 


Cts — max I(^X\^ — Ym j Yi,Ym ) 

p(xi,X2,...,X,ri) 


nN. 


= mx max I{Xi;Y), G {0,1,-}. 

p{x), m 

For a fair comparison of the Cls with Cts-, we consider = Aj^^. 


( 10 ) 

( 11 ) 

( 12 ) 


III. Capacity Upper Bound 

There is no closed form for the Binomial channel capacity. As such, for the first time, we propose an 
upper hound for the Binomial channel at the low SNR regime hy considering power and peak constraints, 
i.e., E[X] < a, 0 < X < A respectively. The Binomial channel is defined as follows: 

P{Y = y\X = x)= (13) 

(^^)/p,(a^ + ^ne)(l - fpSx + , 

fp, : [0,oo] [0,1],?/ G {0, l,...,A^n} 

p{y\x) is hinomial distrihution, we have '^yyp{y\x) = Nnfp^{x). 

We infroduce a new upper hound on fhe capacity of the Binomial channel based on the symmetrized 
Kullhack-Leihler (KL) divergence, referred as KL upper hound in | [TT| . We first explain the KL upper 
hound briefly. Let i2sym(p|k) = D{p\\q) + D{q\\p). Then, 

li{p{y\x)) = maxDsyn,{p{x,y)\\p{x)p{y)) (14) 

p(x) 

> maxI{X;Y) = C{p{y\x)). 

p(x) 
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The KL U{p{y\x)) is always an upper bound on the capacity. It is straightforward to show that 

Dsyrr,{p{x,y)\\p{x)p{y)) = (15) 

'^p{x,y) ^Ogp{Y\X) - Kp(^x)p{y) ^Ogp{Y\X) 

Now, we state our upper bound in the following theorem. The proof is provided in Appendix [A| 

Theorem 1. Consider a point to point Binomial channel P(Y = y\X = x) = {^^)fpb{x + A„e)(l — 
fpb{x + , and any input probability mass function (p.m.f) p{x) where fp^ : [0, oo] —>■ [0,1], y G 

{0,1,..., A^n}. Then, for any input distribution the symmetrized KL divergence upper bound has the 
following explicit formula: 

I{X;Y)<U{p{x,y)) (16) 

= NnCoy{fp,{X + Ane),log{ xJ ), 

JpbK^ ' ^ne)) 

where Cov(X, y) = E[Ay] — E[A]E[y]. Furthermore, for a Binomial channel with average intensity 
constraint a and peak constraint A we have 


^Binomial {p{y\x)) := niax U{p{x,y)) 

p{x): 

E[X]=a,0<X<A 

+ + If {*) 

MA+^E, if {**) 


where E = log 




Pb 

. Hence, 


Pb _ 

fpbi^pp)A 


w = /p.(«+^ne) 


< 


, and (**) = fp,^{a + Ane) > 


C = 


max /(^;y) < Z^Binomial(p(y|2;))- 

p{xy. 

E[X\=a, 0<X<A 


(17) 


We compute this KL upper bound numerically in Section VI Based on numerical evidence, we believe 
that this upper bound works well for Binomial channels (such as MC channels) with low capacity. 


IV. Capacity Lower Bound 

In this section, we compute a lower bound for the Binomial channel when the environment noise is 
negligible, by assuming a binary input, while in the previous section, continuous input was assumed. 
Further, we do not consider the average constraint. We compute a closed form formula for the lower 
bound. As an example, the transition probabilities of the Binomial channel with Am = 0, re = 2, and 
= 2 is shown in Fig. The proof of the following lower bound is provided in Appendix [B| 
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Fig. 3: Binomial Channel, n = 2, N = 2 


Lemma 1. Consider a point to point Binomial channel P(Y = y\X = x) = {^y^fpt{x + Ane){l — 
fpb{x + Ane))^^~^, and any input p.m.f p{x), in which Ane = 0 and x G {0,74^}. A lower bound on 


the capacity of this channel is obtained as: 

dlower 


1 

1 + eAPb) ■ 


gjPb) 

1 + ediPb) 


(18) 


where pb = 9{Pb) = ^ -p\ogp - (1 -p)log(l - p). The capacity of 

this channel is a lower bound to the Binomial channel capacity without the energy constraint, when the 
environment noise is zero. 


If we consider Nn = 1 then the channel would reduce to a Z-channel. 


V. Blocking of Molecule near Receptor 

In the previous sections, for the TS scenario we assumed orthogonal parallel channels for different 
types of molecules where there is no interference between different types of molecules (i.e. no blocking 
of a receptor by molecules of another type). However, it is expected that when there are different types of 
molecules, they may interfere with each other. In other words, one type of molecule may block another 
type of molecule from binding to its receptor pair. For example, consider m = 2 with two types of 
molecule. A, B and their corresponding receptors as Ra, Rb respectively. The molecule type A near Rb 
may prevent the molecule type B from binding to Rb and vice versa (see Fig. [^. 

Assume that Xa and are the received average concentrations of types A and B, respectively. The 
main reaction kinetics, for binding the molecule type B to its receptor may be modeled as Q: 

Xb + Rb^^XRb, (19) 

hiB 
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Fig. 4: Blocking of Ra 





1 — kb ^ \ Full 


(a) Markov Model with no Blocking for Receptor type B 
Kb Q 

I — K.B 0 “'> < Emptyy ^Blockji y 


IbXb 

(b) Markov Model with blocking for Receptor type B 


Fig. 5: Two Markov Models for Receptor type B 


where 7 b > 0 is the association rate of molecules type B with receptors of type B and kb > 0 is 
the dissociation rate of XRb complex. Now, we can characterize the blocking for the receptor type B, 
similar to the reaction kinetics formulas, hy: 


Block,A 

"vz" I Ty 'y B T~)Block.A 

Aa + Bb -> , 


Xa + Rb 


Block,A 


R 


.Block,A 
B 


( 20 ) 


where 7^ ’ > 0 is the blocking rate of iiB by molecule type A and is the unblocking rate 

of ji^ \Yas blocked by molecule type A). If we do not take the blocking into account, 

then we have a reaction kinetic for each type of receptor to its type of molecule. As in Q, we may 
define a Markov model for the no blocking case based on ( [T^ , as shown in Fig. 5a for m = 2. Likewise, 
according to (201 and similar to ([T^, we propose a Markov model for the blocking as shown in Fig. 5b 


We consider three states. The full state is when the receptor binds to its type, the empty state when 
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(a) = 0 





(b) = 5 

Fig. 6: Capacity of LS and TS for OiLS = ^ and ars = ^■ 


the receptor is free, and the block state when the receptor is blocked with a different type of molecule. 


The steady state behavior of the system reaction formula can be obtained as (see Fig. 5a): 

Xb 


Pb = PFuU = 


7b 


( 21 ) 


Solving the chain for the blocking case, we have the following probability of binding and blocking: 


Pb = PFull = 


^Xb 

Kb ^ 


Block. A 

7b 


K^Xb + ^BloaKA 




Block,A 

2b 




PBlock = 




Block,A 

Jb 


Xa + 1 


( 22 ) 


(23) 
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If we increase one type of molecule, the probability of binding for another type is decreased as expected. 
This model can be extended for m > 2 via, 


Pb = PFuU = 


PBlock 




1±Y- -I- if V”* Y - ^ 

K,i ' f^Block 2_^j = \^j^i ' 

7,^'°°'° v^m y- 

^Bloak = 


lfX,+ 




(24) 


(25) 


where p\ and probability of binding for the zth molecule to its receptor and the blocking 

probability of the ith type of the receptor by the another type, respectively. We assume the same blocking 
and unblocking rates for the zth receptor, which are defined by r^Biock i^Biock respectively. It is also 
possible to consider the environment noise Ane for the binding and blocking probabilities. By considering 
the probability of binding when there is blocking, this channel is a multi-input multi-output Binomial 
channel, whose capacity is defined as follow: 


C'j's — max /(2 li, —ki; k^), 
0<Xi<A,,E[Xi\<a 

r.e{o.i . 

m 

— Xi,..., — Xttt,) — 

( nN \ 

y- ■■■^^rn,Ane){l “ /p-(xi, Xm, Ane))^ 


(26) 




where /p*(xi, ...,Xm,Ane) = pi is the probability of binding when the blocking is considered. 


VI. Simulation 


In this section, we evaluate the rates of TS scenario given in equation ( [T^ , and the LS scenario given 
in equation (|9]), using Blahut-Arimoto algorithm (BA) 112|. We consider n = 16, iV = 5 ,7 = 0.0004 and 

K = 0 . 1 . 


Fig. 6 a shows the capacity for LS and TS, for m = 2,4, 8,16 when = 0. It is seen that 


increasing the number of molecule types, m, from 1 improves the performance (for fixed A^), which 
is expected due to the parallel transmission of the molecules. However, if we continue to increase m, 
and accordingly decrease the number of bacteria in each colony to n/m, the performance degrades. The 
reason is that decreasing the concentration level of TS in equation ( fTO] ) decreases the binding probability. 
Hence, there is an optimal m. For example, we can see that, for Ag = 80, this optimal value lies between 
m = 4 and m = 8. This means that for = 80, at m = 2,4 the TS capacity is higher than the LS, but 
for m = 8,16, it is lower than the LS. Similar conclusions can be made from Fig. I^in the presence of 
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the environment noise = 5 . Our proposed KL upper bound, in (171, is depicted in Fig. 

where the gap between the capacity and the upper bound decreases as the environment noise increases. It 
can be observed that the distance between the KL upper bound and the capacity is constant in logarithm. 



Fig. 7: Capacity and KL upper bound in terms of Ane for the Binomial channel with Ag = 80 and 
a = 40. 

The lower bound in ( [T^ along with the capacity are shown in Fig. For small values of Ag, our lower 
bound is tight which means the binary distribution is capacity achieving distribution for small values 
of ^4^. Fig. 1^ shows the effect of blocking by showing the capacity of LS and TS for m = 2. We 
assumed _ ^Biock _ q.OOOS, = 0.01, 71 = 72 = 0.0004 and ki = K 2 = 0.1. 

As illustrated, blocking decreases the capacity of TS. In this case, the loss due to blocking is even more 
than the improvement due to the use of multiple molecule types. 

VII. Conclusion 

In this paper, we first investigated capacity performance of type and level scenarios. Next, we derived a 
new upper bound for the capacity of the Binomial channel at low SNR-regime based on the KL-divergence 
bound as well as a lower bound. Next, blocking was modeled as a Markov process and the probabilities 
of binding and blocking were derived. As expected and confirmed by simulations, the blocking would 
decrease the capacity of type scenario. 
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Fig. 8: Capacity and Lower Bound in terms of ^4^ for the Binomial channel with n = 4 and N = 5. 



Fig. 9: Capacity for TS hy considering blocking and without blocking and LS for ciLS = ^ and 
ars = 4^, for Ane = 0. 
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Appendix A 
Proof of Theorem[I] 

We find KL upper bound for Binomial channel as follows: 


HX-.y)<Y, [p(x, y) - p{x)p{y)] \ogp{y\x) 
x,y 

= ^[p(x,y) -p{x)p{y)] 
x,y 

log + A„e)(l - /p.(x + A^,))^^-y) 


'^\p{x,y) - p{x)p{y)][log 




Nn 


+ ?/log/p,(x +A 

ne) 

+ {Nn - y) log (1 - fp, {x + Am))] 

^p{x,y) [loS 


Nn 

y 


Nn 

y 


^p{x)p{y) 

+ Ep{x,y)[y log/p,(x +A ne)] 


^p{x)p{y) \y fpb ^ne)] 

+ Ep(^x,y) [{Nn - y) log (1 - fp, {x + A„e))] 

- Ep^^)p(^y)[{Nn - y) log (1 - fp,{x + A^))] 
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= Ep(^x,y)[y'^Ogfp,{x + A ne}] 


Ep(^x)p{y) [y fpb E ^ne)] 

- [Ep{x,y) [y log (1 - fp, (X + Ane))] 

- ^p(x)pfo)[2/log {1 - fp,{x + Ane))]] 


= Ep(^x,y)[y'^Og 


fpb E Ane) 


^ fpb ^ne) 

TP \ 1 fpb{x-\- Ane) 1 

gl-/p.(x + ^ne)] 

fpb E Ati 




X y 


fpb E Ane) 

fpb {x E Ati 


:E(E ypiy\x))p{x)) {^log- 


X y 


fpb E Ane) 


P{x)) 


= E[Nnfp, {x + Ane) log ( J ] 

■*■ Jpb ' -^ne) 

- ElNnf^Jx + A„,)|E[log( . )] 

Jpb\^ ' -^ne) 

= NnCo^{fpSX + Ane), \og{ ^J )• 

V-*- JPb + ^ne)) 


As mentioned earlier, fpi^{X+Ane) is an increasing function. Hence the Co^{fp^{X+Ane), log( )) )) — 

0. A further ohservation is that 


C < max7VnCov(/p,(X + Ane),log( nJ ) 

pE) - Jpb{E- + Ane)) 

is always achievable with a binary rv X. We consider two points, xi and X 2 with probabilities pi,P 2 - 

We have 


maxCov(/p,(A + 

^ne); log(F)) 

p{x) 


max E[fp^ {X + Ane) log F] - 

pE), 

E(fpb (X+A„e))<Q',0<X<A 

E[fp,iX + Ane)]E[logF] 

max E[{fp^{X + Ane)- 

pE), 

E(fpb {X+Ar,,))<a',0<X<A 

E[fp,iX + Ane)])\ogF] 
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where F = ^ ^ ■ Now, based on the analysis in | |II| Appendix C], the optimal distribution i 

given by p{x) = y i^a+a “ ^) + (1 “ / {A+A ) )^(^) upper bound is obtained as: 


a 


i- / . , — A —f 

'</p(, (“+^Jie) fpti^ F Ajie) 


[fpbi^ + ^ne) — a]F 


where E = log upper bound is equal to 


/pt (j4pe)(l —/pb (A+Aric)) 

fpbiol + A. 


for a' < 


fpb {A + Ar, 

and 


\fpb E A^ie) fpi^ [ex + A^e)] loS 


fphA+^ric) fp^{A+A„^) /p^(7t+A„,)(l-/pA^Pe)) 


2 “““ 4 

Now if we consider fp^{X + Ane) = x+A 


/pAA„,)(l-/,AA+^Pe)) 

— , then the upper bound is: 


fpbjA + Ane){l — fp^{Ane)) 
/p6(Ane)(l — fpb{A + Ane)) 

Otherwise. 


^Binomial(p(2/|®)) ■ 


max 

p{x): 

E[X]=afl<X<A 


U{p{x,y)) 


= nN < 


fj’lA+xC^) [fpbi^ + ^ne) — fpbi'A + Ane)]E, if, (= 1 =) 


iPb 

fph {A+A„Pj 


E. 


if [**) 


where, E = log (a1k"i-/p, (a+aT ])] ’ ^ _|_ ^ 

fp^{A+A„,,) 


Let’s define: 


Appendix B 
Proof of lemma[I] 


H^iPb) = 

Pc=il-Pb)^^ 


Then, the binomial channel transition probabilities by considering binary input, is characterized by: 


p{y = 0|x = 0) = 1, p{y = i|x = 0) = 0, i = 1,..., nN, 
p{y = i\x = 1) = - Pb)"‘^~\ i = 1, ...,nN, 


DRAFT 



















18 


Assume p{x = 0) = a and p{x = 1) = 1 — a. The lower bound on Binomial channel capacity could be 
derived as below: 

Qon^er = - H{Y\X) 

a 

1 

= max H(Y) — H(Y\x)p(x) 

x=0 

= maxH{Y) - (1 - a)H{Y\x = 1) 

Q; 

nN 

= max — > ((1 — oc)p(y = i\x = 1)) log ((1 — a)p{y = i\x = 1)) 
i=l 

- (a + (1 — a)pc) log (a + (1 — a)pc) — (1 — a)H{Y\x = 1) 

nN 

= max — > ((1 — Oi)p{y = i\x = 1)) log ((1 — a)p{y = i\x = 1)) 
i = l 

- (a + (1 - a)pc) log (a + (1 - a)pc) - (1 - a)H"'^{pb) 

= max-(l - a)(l -pc)log (1 - a) + (1 - a)pclogpc 
a 

+ (1 - a)H"'^{pb) - (a + (1 - a)pc) log (a + (1 - a)pc) 

-{l-a)H^^{pb) 

= max-(l - a)(l -pc)log (1 - a) + (1 - Q;)pclogPc 
a 

- (a + (1 - a)pc) log (a + (1 - a)pc) 


Taking a derivative with respect to a and setting to zero we obtain: 

1 


a = 


1 J_ ± _ 

I —Pc log Pc 

e i-Pc —p^ 


Then 


Clower 


1 


g{pc 


1 _|_ es(pc )' 1 _|_ e9(pc) 


where g{pc) = and H{p) = -p\ogp - {I - p) log(l - p). 
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